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Summary 


Most state departments of education across the United States recommend or require that 
districts use a home language survey as the first step in a multistep process of identify- 
ing students who qualify for English learner student services (U.S. Department of Educa- 
tion, 2012; Zehr, 2009). School districts typically administer the home language survey to 
parents and guardians during a student’s first enrollment in a school or district to deter- 
mine whether the student speaks a language other than English or whether parents or 
guardians speak a language other than English to the student in the home (Linquanti & 
Bailey, 2014). Parents, educators, and the U.S. Department of Education Office for Civil 
Rights have raised concerns that existing home language surveys may not reveal accurate 
information about students’ language skills or exposure to English language and literacy 
because of inconsistency in administering these surveys, among other reasons. Therefore, 
the home language survey may contribute to the misidentification of English learner stu- 
dents (Bailey & Kelly, 2010). 

Regional Educational Laboratory Northeast & Islands, in partnership with the English 
Language Learners Alliance, developed the Home Language Survey Data Quality Self- 
Assessment to help state departments of education and school districts improve the quality 
of data collected through home language surveys and improve the identification of poten- 
tial English learner students. The 44-item self-assessment is designed for state leaders 
who coordinate district programs to support students’ English language acquisition and 
achievement, as well as for district leaders who oversee the process of identifying English 
learner students in schools. The self-assessment is designed to be used by state staff to 
gather information from district English learner program coordinators on four areas that 
affect the quality of data collected through home language surveys: purposes, policies, and 
guidelines about home language survey administration; data collection practices; personnel 
support; and data management. The self-assessment also prompts district staff to self-assess 
key practices in home language survey development and administration to improve high- 
quality data collection at the local level. 

This report presents the Home Language Survey Data Quality Self-Assessment, along 
with instructions on how to administer it, and describes how to engage stakeholders — 
including state departments of education, district English learner program coordinators, 
school English learner specialists, parents, student enrollment/registration staff, and data 
managers — in analyzing and interpreting its results. Reflecting on the results at both the 
state and district levels can inform decisions that could contribute to more accurate data 
on English learner students and more accurate resource allocations for districts. 
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What is the Home Language Survey Data Quality Self-Assessment? 


Most state departments of education across the United States recommend or require that 
districts use a home language survey as the first step in a multistep process of identifying 
students who qualify for English learner student services (U.S. Department of Education, 
2012; Zehr, 2009). Home language surveys are typically completed by parents or guardians 
and are intended to determine whether a student speaks or is spoken to at home in a Ian- 
guage other than English. The aim is to identify potential English learner students rather 
than to measure English language proficiency. Depending on the results of the survey, stu- 
dents may then undergo assessment of their English language proficiency to determine 
whether they are entitled to receive specialized language and academic support services as 
English learner students (Lhamon & Gupta, 2015; Linquanti & Bailey, 2014). 

In partnership with the English Language Learners Alliance, Regional Educational Lab- 
oratory (REL) Northeast & Islands developed the Home Language Survey Data Quality 
Self-Assessment to help state departments of education and school districts improve the 
quality of data collected through home language surveys. The self-assessment was original- 
ly developed to support a small working group of state and district leaders in Connecticut 
and Rhode Island in identifying and addressing challenges to the quality of their home 
language survey data (see appendix A for further information on the process of developing 
the self-assessment). That original work has been revised and expanded so that the current 
self-assessment survey can support states in improving the quality of home language survey 
data as part of an effort to more accurately identify potential English learner students and 
allocate resources to students who qualify for them. 


The Home 
Language Survey 
Data Quality Self- 
Assessment is 
designed to help 
state departments 
of education and 
school districts 
improve the quality 
of data collected 
through home 
language surveys 


The self-assessment is based on a data quality framework, which is supported by current 
research and expertise from the field. The framework describes four areas that affect the 
quality of data collected through home language surveys: purposes, policies, and guidelines; 
data collection practices; personnel support; and data management. The 44 assessment 
items are organized to gather information on the district context and then on district prac- 
tices in each of the four framework areas (table 1). Further description of the framework is 
provided in appendix B. The matrix in appendix C illustrates the alignment of assessment 
items with the four areas represented in the framework. 


State departments of education can choose to administer the 15-minute self-assessment 
to all school districts or to a subset of districts. Districts may also choose to complete the 
self-assessment independent of any state study in order to identify strengths and weakness- 
es in local home language survey practices. The self-assessment is intended to be complet- 
ed by the district English learner program coordinator 1 or the individual whom the district 
identifies as the most knowledgeable about how the home language survey is administered 
locally. 


Why administer this self-assessment? 


Although the state- or district-created home language survey is the first step in a multistep 
process of identifying English learner students, recent studies point to multiple factors that 
can undermine the quality of home language survey data. These factors include unclear 
survey purposes, 2 lack of appropriate language support for parents to complete the survey, 
and inconsistent practices during survey administration (Bailey & Kelly, 2010, 2013; 


1 


Table 1. Self-assessment items, by section 

1 Item number 

Section 

Guiding question and description of items j] 

1-6 

1. District context 

District size, English learner student population, and means of 
home language survey administration. 

7-18 

2. Purposes, policies, 
and guidelines 

Has the district established clear purposes, procedures, and 
guidelines to guide high-quality data collection during administration 
of the home language survey? 

Items address the specification of procedures and expectations that 
promote common understandings and practices for home language 
survey administration. 

19-28 

3. Data collection 

practices 

Does the district ensure that its home language survey policies and 
guidelines are applied during design and administration of the home 
language survey? 

Items address the application of prescribed procedures and 
expectations, typically enacted as a sequence of events using 
provided resources. 

29-32 

4. Personnel support 

Does the district ensure that personnel who administer the home 
language survey have the necessary knowledge, skills, and support 
to collect high-quality data? 

Items address the preparation and supervision of personnel to 
ensure the necessary knowledge and skills to apply the prescribed 
procedures, expectations, and resources for home language survey 

administration. 

33-42 

5. Data management 

Does the district ensure that data management systems and 
practices contribute to high-quality data? 

Items address the use of systems, procedures, and expectations for 
entering, archiving, and reporting home language survey data. 

43-44 

6. Closing questions 

Questions about consistency of administration and perceptions 
about accuracy of results. 

Source: Authors' compilation. 


Linquanti & Bailey, 2014). These factors may contribute to poor data quality, which can 
result in the misidentification of potential English learner students in school districts. Mis- 
identifying students poses a challenge for both districts and state education departments 
when they allocate resources to support English learner students’ success. 

Benefits for state departments of education 

State departments of education can use the Home Language Survey Data Quality Self- As- 
sessment to learn more about the variation in how home language surveys are administered 
across their state and to identify ways to target resources and support toward collecting 
accurate survey data. State support might include developing online training modules to 
ensure broad understanding of the purpose and use of the home language survey, creating 
a sample home language survey administration script, or providing home language survey 
forms translated into a range of languages spoken by parents across the state. By analyzing 
results from the self-assessment, state leaders can gain a better understanding of how to 
help districts collect accurate data at the start of the English learner identification process. 

Benefits for school districts 

For school districts, completing the assessment can prompt reflection among respondents 
about the importance of home language survey data, facilitate the review of factors that 


2 




affect data quality, and suggest ways to take action to improve the quality of the data 
collected. When respondents note that these activities do not occur or occur to a limited 
extent, they also gain practical ideas about ways that their local survey practices might be 
strengthened, because each survey item provides potential directions for improvement. For 
example, items ask about the extent to which those who administer the home language 
survey receive training in specific topics, the extent to which parents are informed of par- 
ticular issues, and the extent to which language-based support is provided. English learner 
program coordinators who reviewed the self-assessment during its development indicated 
that when such practices were not occurring in their districts, the items suggested useful 
practices they could pursue to strengthen the quality of their data. 


How to administer the self-assessment 


State and district leaders can access the Home Language Survey Data Quality Self-Assessment 
in this document. The self-assessment includes an explanation and directions for each section, 
definitions of several terms, and items and their respective response choices. The Home Lan- 
guage Survey Data Quality Self-Assessment should not be administered when states are in 
the midst of revising their home language survey forms or guidance. Doing so may lead to 
respondents conflating current district practices and anticipated district practices once state 
guidance is finalized. Such mixed responses may make it impossible to draw valid conclusions 
from the results. The self-assessment should take 15 minutes for respondents to complete. 

Administration by state departments of education 

State departments of education may choose to administer the self-assessment to all school 
districts or to a subset of districts. English learner program coordinators (or the individu- 
als most knowledgeable about how the home language survey is administered locally) are 
often best prepared to complete the self-assessment. States should explain to respondents 
the purpose of the self-assessment and the state’s intended use of the data. States can model 
the explanation after the information included in the introduction of the self-assessment 
but should add relevant information about home language survey data use in their state 
context. States should also revise the welcome page of the self-assessment to convey how 
they will use the results, revise definitions to reflect current state definitions, and modify 
the closing page to include relevant contact information. 


State departments 
of education can 
use the Home 
Language Survey 
Data Quality 
Self-Assessment 
to learn more 
about the variation 
in how home 
language surveys 
are administered 
across their state 
and to identify 
ways to target 
resources and 
support toward 
collecting accurate 
survey data 


States may choose to administer the self-assessment in paper-and-pencil format or through 
an online survey platform. An online platform allows the self-assessment administrator 
to require a response to closed-ended items before the user can proceed to the next item. 
Requiring responses can assist in reducing item nonresponse bias. 3 An online platform 
also facilitates the efficient compilation of results and often includes easy-to-use reporting 
features that produce graphical displays, tabular counts, or response distributions. 


Results from online survey platforms can be downloaded or exported for subgroup or cross-tab- 
ular analysis, which can help states understand how and why the quality of home language 
survey data varies across districts. To support a high rate of return for the self-assessment, and 
thereby reduce survey nonresponse bias, it is recommended that states provide an electronic 
link to the self-assessment and time to complete it during a regularly scheduled statewide 
meeting (for example, a quarterly meeting or professional development session for English 
learner program coordinators) or as a required component of a state email communication. 
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Administration by district leaders 


District leaders or teams may decide to independently complete the self-assessment in order 
to assess and reflect on the strengths and weaknesses of their district’s local home language 
survey practices. In this case, respondents can disregard the first paragraph of the welcome 
text in the self-assessment. The self-assessment items can be completed individually by the 
district leader or, to increase reliability, collectively with a team. To determine priorities for 
improving data quality, respondents can reflect on the items that indicate areas of strength 
(practices that occur to some or great extent) or weakness (practices that occur to little or 
no extent). 


How to analyze the self-assessment results 


State departments of education and district leaders can analyze the results of the Home 
Language Survey Data Quality Self-Assessment to identify priorities for improving home 
language survey data quality. States can then use these priorities to develop evidence-based 
policies and guidance and to strengthen resources and support for districts related to devel- 
oping and administering home language surveys. District leaders can also use the results to 
identify areas of weakness in their local practices. 

Analyzing results at the state level 

The first step in analyzing self-assessment results from multiple districts across a state is 
to create a set of basic descriptive statistics that can be used to determine how many and 
which types of districts are represented. For example, summary statistics could include the 
overall response rate and statistics determined by items from the “Your district context” 
section: the percentage of respondents that represent small, medium, and large districts 
(item 1); the percentage of respondents from districts with a low, medium, and high inci- 
dence of English learner students in their population (item 2); the percentages of respon- 
dents from districts that administer the home language survey through an online platform 
or by paper and pencil (item 5); and which staff typically administer the survey (item 6). 
In addition, it may be useful to know the percentage of respondents by role (item 3) and 
the percentage of respondents who report having an accurate understanding of the state’s 
home language survey policy (item 4). 


State departments 
of education and 
district leaders can 
analyze the results 
of the Home 
Language Survey 
Data Quality Self- 
Assessment to 
identify priorities 
for improving home 
language survey 
data quality 


In addition to calculating frequency distributions that describe the respondents and their 
local contexts (items 1-6), users can calculate or visually represent the frequency distribu- 
tions of responses to the remaining items in sections 2-5 (see table 1). For example, the 
percentage of respondents who indicated that a particular home language survey practice 
occurred “to no extent,” “to little extent,” “to some extent,” and “to great extent,” as well as 
the percentage who indicated “I don’t know” will need to be reviewed. 


Summary statistics serve as a launching point for interpreting the self-assessment results. 
For example, if the target population was all districts, but most of the responding districts 
are small and midsized districts, the results may not necessarily reflect the perspectives 
of larger districts in the target population of the self-assessment administration. Further, 
summary statistics may generate questions about how the quality of home language survey 
data varies across different types of districts. 
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When examining the self-assessment data, it is recommended that those interpreting the 
results decide together the level at which a statistic is deemed acceptable or concerning, 
based on knowledge of their local contexts. For example, users in one state might discuss 
self-assessment results and agree that response rates of 70 percent or greater indicating “to 
some extent” and “to great extent” will be considered an area of strength or that response 
rates of 30 percent or greater indicating “to little extent” and “to no extent” will indicate 
an area of weakness. Users in another state might use different cutpoints. 


Determining statistical cutpoints is only a starting point for more substantive conversations 
about the meaning and potential causes of particular results. Through these conversations, 
teams analyzing the self-assessment collectively explore and establish their priorities for 
improvement. Their final priorities may or may not adhere to the parameters they initially 
set for determining areas of strength and weakness. 

To support these interpretations, users should compile results from open-response items 
from the end of each section and group those comments thematically (for example, by 
type of comment, question, or suggestion). These qualitative data can provide important 
clarifications, ideas for improvement, or experiences that may prompt more specific or 
in-depth analysis. 

Analyzing results at the district level 

At the district level a single individual or team typically completes the Home Language 
Survey Data Quality Self-Assessment; only when there are multiple respondents is it nec- 
essary to create summary statistics. Most important, the self-assessment serves as a tool for 
guided reflection. The district pattern of responses may suggest actions that can strengthen 
the quality of home language survey data. 


Stakeholders 
can be invited to 
investigate how 
home language 
survey practices 
might be affecting 
data quality 
and subsequent 
identification of 
English learner 
students and what 
actions can be 
taken to improve 
data quality 


For example, items in the second section ask about specific practices that can be imple- 
mented to reinforce district guidelines, such as conveying the purposes and uses of the 
home language survey to parents and explaining how confidentiality will be maintained. 
Indications by respondents that topics are not currently addressed or are addressed only to 
a little extent should prompt discussion, sharing of observations, and consensus building 
for action, if they are established as priorities. The content of the items suggests direction 
for improvement. Similarly, each section suggests a range of specific district actions for 
improvement. 

How to engage stakeholders to drive improvement in data quality 


State and district leaders can engage stakeholders in reviewing the results and determining 
the actions that can be taken to address data quality concerns. In particular, stakeholders 
can be invited to investigate how home language survey practices might be affecting data 
quality and subsequent identification of English learner students and what actions can be 
taken to improve data quality. 

Both state departments of education and district leaders can refer to the data quality frame- 
work (appendix B) and Workshop protocol: Working with the results of the Home Language 
Survey Data Quality Self-Assessment (appendix D). The data quality framework provides a 
big-picture look at the purpose of the self-assessment. The protocol describes a workshop 
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session and provides content and process resources for facilitators to engage a diverse group 
of stakeholders in examining and using the self-assessment results to drive improvements. 
The protocol is designed for state departments of education, which may benefit from invit- 
ing a selection of district English learner program coordinators, school English learner spe- 
cialists, parents, student enrollment/registration staff, and data managers into the process. 


At the district level, English learner program coordinators can also benefit from inviting 
knowledgeable stakeholders, such as district student enrollment/registration staff, parents 
and parent liaisons, and school-based English learner specialists, into the discussion of the 
self-assessment items and collective response to them. A facilitator can modify the work- 
shop protocol to help participants interpret the results, begin to establish consensus around 
priorities, and build commitment to improving the quality of home language survey data. 

As detailed in the workshop protocol, session facilitators should do the following to engage 
diverse stakeholders in analyzing the data and discussing and prioritizing actions: 

• Explain the purpose of reviewing the results of the Home Language Survey Data 
Quality Self-Assessment, particularly in relation to improving state or district pol- 
icies, guidance, and support for the home language survey. 

• Discuss the data quality framework as a facilitating structure for understanding 
factors that affect the quality of home language survey data. 

• Facilitate the group’s examination of the self-assessment results to identify 
strengths and weaknesses in guidance, data collection, personnel support, and 
data management related to the home language survey. 

• Ensure that the group identifies and prioritizes actions that will advance the quality 
of home language survey data and that the group articulates key lessons that can 
inform the quality of other efforts to collect data on English learner students. 

For both state and district leaders, including diverse stakeholders in this work could expand 
the perspectives and experiences being considered, deepen discourse and problem solving, 
and engage additional expertise and commitment in addressing prioritized areas of need. 


For users 
interested in 
altering, adding, 
or removing self- 
assessment items 
to address state or 
district contexts, 
it is important to 
understand the 
information that 
the current items 
are intended to 
collect and how 
they can point to 
distinct areas of 
a system that may 
need attention 


How to adapt the self-assessment to specific state or district contexts 


Users may be interested in altering, adding, or removing self-assessment items to address 
their particular state or district contexts. But to effectively identify areas in need of 
improvement, adaptations should be aligned with the data quality framework. The matrix 
in table Cl in appendix C can assist with decisionmaking on adaptations. For example, 
the matrix illustrates that 11 items in the self-assessment address the purposes of the home 
language survey, but the questions vary according to the framework area being addressed. 

Having a variety of questions in each area is not meant to be redundant; rather, items 
are deliberately aligned to different areas of the framework. It is thus important for those 
who are considering adapting the self-assessment to understand the information that the 
current items are intended to collect and how they can point to distinct areas of a system 
that may need attention. Consulting the matrix can enhance this understanding and 
show how adaptations may complement existing self-assessment items. Overall, adapta- 
tions should add value to the full set of items in the self-assessment and provide actionable 
information that helps answer the guiding question for each framework section, as shown 
in table 1 and in table Cl in appendix C. 
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The Home Language Survey Data Quality Self-Assessment 


Welcome! 

Your state department of education is surveying districts as part of a systematic review of 
factors that may affect the quality of data on English learner students (see box for defini- 
tions of key terms). This district self-assessment focuses on data quality related to a common 
first step for identifying potential English learner students — the home language survey. 

This self-assessment should be completed by the person who is most familiar with home 
language survey administration practices in your district. This is often the English learner 
program director but might also be the superintendent, deputy director, accountability or 
compliance manager, or student enrollment coordinator. If you are not the person most 
familiar with home language survey administration in your district, please convey this to 
your supervisor so the survey can be forwarded to the appropriate person for completion. 

Your task. This self-assessment asks you to make a professional judgment about the extent 
to which certain conditions and factors are in place to support the collection of high-quality 


Key terms 

Confidentiality. The obligation of those who receive personal information about an individual to 
respect the individual’s privacy by safeguarding the information. 

Cultural competency. A set of aligned behaviors, attitudes, and policies that come togeth- 
er in a system, agency, or among professionals and enable effective work in cross-cultural 
situations. 

Data quality. The degree to which data provide accurate, clear, complete, and unbiased infor- 
mation; are useful for intended purposes; and are kept secure and protected from unautho- 
rized access or revision. 

English learner student. A student who is not yet able to communicate fluently or learn effec- 
tively in English, who often comes from a home and background based on a language other 
than English, and who typically requires specialized or modified instruction in both the English 
language and in academic courses. 

Potential English learner student. A student identified for further screening to determine 
English learner student status. In most states a student is identified as a potential English 
learner student by interpreting data collected through the home language survey, which is the 
first step of a multistep identification process. 

Source: Confidentiality and cultural competency: Cross, T., Bazron, B., Dennis, K., & Isaacs, M. (1989). To- 
wards a culturally competent system of care (Vo I. 1). Washington, DC: Georgetown University Child Development 
Center, CASSP Technical Assistance Center, http://eric.ed. gov/?id=ED330171. Data quality: U.S. Department 
of Education, Office of Elementary and Secondary Education. (2006, April). Improving data quality for Title I 
standards, assessments, and accountability reporting: Guidelines for states, LEAs, and schools (Non-Regulatory 
Guidance). Washington, DC: Author. Retrieved March 14, 2016, from https://www2.ed.gov/policy/elsec/guid/ 
standardsassessment/nclbdataguidance.doc. English learner student: under English-language learner, in Great 
Schools Partnership (2013). The glossary of education reform. Portland, ME: Author. Retrieved March 14, 2016, 
from http://edglossary.org/english-language-learner/. Potential English learner student: Linquanti, R., & Bailey, 
A. (2014). Reprising the home language survey: Summary of a national working session on policies, practices, and 
tools for identifying potential English learners. Washington, DC: Council of Chief State School Officers. Retrieved 
March 14, 2016, from http://www.ccsso.org/Documents/2014/CCSS0%20Common%20EL%20Definition%20 
Reprising%20the%20Home%20Language%20Survey%2001242014.pdf. 
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data through the administration of the home language survey. You may want to refer to a 
copy of your home language survey while completing this self-assessment. After a few initial 
background questions, the core of this self-assessment tool should take approximately 15 
minutes to complete. Items marked with an asterisk (*) are required. 

You should consider only your district’s guidelines, data collection processes, personnel 
support, and data management procedures for the home language survey when responding, 
rather than the full multistep procedure for identifying English learner students. 

Results. Members of the state department of education will review these data and may 
share this information with districts to support improved data quality. No information 
will be shared about individuals or individual districts. Assessment results will inform your 
state’s efforts to support more accurate identification of English learner students to ensure 
that these students receive appropriate services and support. 

1. Your district context 

First, a few brief questions about your district context. Items marked with an asterisk are 
required. 

*1. Approximately how many students are enrolled in your district? 

□ Fewer than 3,000 

□ Between 3,000 and 10,000 

□ Between 10,001 and 25,000 

□ More than 25,000 

□ I don’t know 

*2. What percentage of your district’s students are identified as English learner students? 

□ 0-10 percent 

□ 11-20 percent 

□ 21-30 percent 

□ 31-40 percent 

□ 41-50 percent 

□ More than 50 percent 

□ I don’t know 

Your district role 

*3. What is your primary role? 

□ Superintendent/executive director 

□ Assistant superintendent/deputy director 

□ Accountability/compliance manager 

□ English learner program director or coordinator 

□ Student enrollment/registration coordinator 

□ Data specialist 

□ Teacher 

□ Other (please specify) 
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Your home language survey context 


*4. Which best describes your understanding of your state policy for the home language 

survey? 

□ My state mandates the use of a home language survey and provides the survey 
items, which districts are required to use without modification. 

□ My state mandates the use of a home language survey and provides required 
survey items, but districts may include additional questions. 

□ My state mandates the use of a home language survey and provides sample 
survey items, which districts may adopt, modify, or substitute for with their own. 

□ My state mandates the use of a home language survey but does not provide 
districts with sample survey items, so districts create their own. 

□ My state does not mandate the use of a home language survey but provides 
sample survey items for districts to use. 

□ My state does not mandate the use of a home language survey and does not 
provide sample survey items for districts to use. 

□ I don’t know whether my state mandates elements of our home language survey. 

□ Other (please specify) 

*5. In what form is your district’s home language survey administered? Select all that 

apply. 

□ Paper survey (including a form downloaded from district website) 

□ Online survey via computer, tablet, or other electronic device 

□ Verbal survey administration for all families 

□ Verbal survey administration for select families (for example, low literacy, non- 
English speaking, disability) or if requested 

□ Other (please specify) 

*6. Who typically administers your district’s home language survey? Select all that apply. 

□ District licensed/certified professional (for example, English learner program 
coordinator, bilingual specialist) 

□ District support staff (for example, secretary, school enrollment/registration 
personnel) 

□ School licensed/certified professional (for example, English learner teacher, 
pedagogue, counselor) 

□ School support staff (for example, secretary, parent/community liaison, 
paraprofessional) 

□ Parents complete the form independently (for example, the district provides a 
link to an online home language survey and parents complete it at home) 

□ Other (please specify) 
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Directions 


For most of the following items, you are asked to respond based on the extent to which it 
occurs: 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

You may also respond “I don’t know.” 

You may add comments, questions, or suggestions at the close of each major section. Your 
comments provide important information about your local context and role and will help 
your state interpret the self-assessment results. 

2. Purposes, policies, and guidelines 

The questions in this section ask about your district’s purposes, policies, and guidelines for 
the home language survey. 

Guidelines may address the purposes and uses of home language survey data, procedures 
and expectations that guide data collection and data management, and timelines and pro- 
cesses for supervision and support of those who administer the home language survey. 

Guidelines might be conveyed in written documents, visual displays, verbal explanations 
from an administrator or coordinator, informal communications or anecdotal notes, or 
other forms. 

Data quality can be affected by the clarity, consistency, and availability of district guid- 
ance to those administering the survey. 

*7. Has your district established guidelines for administering the home language survey? 

□ Yes 

□ No (If “No,” skip to question 15) 

□ I don’t know (If “I don’t know,” skip to question 15) 

*8. In what ways does your district convey guidelines for administering the home lan- 
guage survey? Select all that apply. 

□ Written documents (for example, handbook or procedural document) 

□ Visual display (for example, posters) 

□ Verbal explanation from an administrator or coordinator 

□ Informal communications/anecdotal notes or records 

□ None 

□ Other (please specify) 
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*9. To what extent do your district’s guidelines clearly address the purposes and intended 
uses of the home language survey (for example, identifying potential English learner 
students for subsequent assessment to determine English learner student status)? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*10. To what extent do your district’s guidelines clearly address what the purposes and 
intended uses of the home language survey are not (for example, determining immi- 
gration status or predetermining the provision of education services)? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*11. To what extent do your district’s guidelines clearly address what to do when an error, 
such as discrepant or omitted data, is discovered in data from the home language 
survey? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*12. To what extent do your district’s guidelines clearly address how and when data from 
the home language survey will be reported to school, district, and, if applicable, state 
offices? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*13. To what extent do your district’s guidelines clearly address procedures and responsi- 
bilities for protecting the confidentiality of data from the home language survey? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 
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*14- To what extent do your district’s guidelines clearly address how to interpret responses 
on the home language survey to identify potential English learner students? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*15. To what extent does your district have translations of the home language survey in 
every language needed by survey respondents? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*16. To what extent are your district’s translations of the home language survey created or 
vetted by professional translators? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*17. To what extent does your district specify a systematic process for periodic review of 
the quality of data from the home language survey? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

18. Before moving on, are there comments, questions, or suggestions you would like 
to make about your district’s policies and guidelines for the home language survey? 
(optional) 
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3. Data collection practices 


The questions in this section ask about data collection practices with the home language 

survey. Aspects of data collection that can affect data quality include survey design, the 

information provided to survey respondents, and data collection procedures. 

*19. To what extent does your district ensure that respondents are informed about the 
purposes and uses of the home language survey? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*20. To what extent does your district ensure that respondents are informed about what 
the purposes and intended uses of the home language survey are not (for example, 
determining immigration status or predetermining the provision of education 
services)? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*21. If the home language survey is embedded in a larger intake or registration packet, to 
what extent does your district distinguish the purpose of the home language survey 
from the purpose of the other registration or intake materials? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

□ Home language survey is not embedded in a larger intake packet 

*22. To what extent does your district ensure that respondents to the home language 
survey are informed about the multiple steps in the process of identifying English 
learner students? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*23. Does your district provide personnel who administer the home language survey with 
a frequently asked questions (FAQs) sheet to accompany the home language survey 
form? 

□ Yes 

□ No 

□ I don’t know 
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*24. Does your district provide personnel who administer the home language survey with 
standardized, simple-to-use scripts to verbally introduce and administer the survey? 

□ Yes 

□ No (If “No,” skip to question 26) 

□ I don’t know (If “I don’t know,” skip to question 26) 

*25. To what extent does your district ensure that administration scripts are read aloud by 
personnel with sufficient foreign language fluency for respondents who need informa- 
tion translated? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*26. To what extent does your district ensure that respondents are provided with a trans- 
lation of the home language survey at the time of completion, as needed? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*27. To what extent do any school or district personnel need to follow up with respondents 
to clarify or complete information submitted on your district’s home language survey? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

28. Before moving on, are there comments, questions, or suggestions you would like to 
make about your district’s data collection practices for the home language survey? 
(optional) 
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4. Personnel support 


The questions in this section ask about the extent to which personnel are prepared and 
supported to administer the home language survey as well as to manage and report survey 
data. The extent, frequency, and consistency of training and support all affect the quality 
of the data collected through the home language survey. 


*29. To what extent does your district ensure that personnel who administer the home 
language survey have adequate and current knowledge about the following topics? 



To no 
extent 

To little 
extent 

To some 
Extent 

To great 
extent 

1 don’t 
know 

a. The purpose or purposes of the home 
language survey 

□ 

□ 

□ 

□ 

□ 

b. What the purpose and use of the home 
language survey are not (for example, 
not for determining immigration status or 
predetermining the provision of education 
services) 

□ 

□ 

□ 

□ 

□ 

c. How to use home language survey data as 
one step in a larger, multistep process for 
identifying English learner students 

□ 

□ 

□ 

□ 

□ 

d. The importance of high-quality data on 
English learner students for allocating 

resources and services 

□ 

□ 

□ 

□ 

□ 

e. Administration procedures of the home 
language survey 

□ 

□ 

□ 

□ 

□ 

f. Cultural competency, or aligned behaviors, 
attitudes, and policies that enable 

effective work in cross-cultural situations 

□ 

□ 

□ 

□ 

□ 

g. Assessing respondents’ needs for written 
translation or verbal interpretation support 

□ 

□ 

□ 

□ 

□ 

h. Confidentiality of home language survey 

responses 

□ 

□ 

□ 

□ 

□ 

i. Data entry, management, and reporting 
responsibilities 

□ 

□ 

□ 

□ 

□ 

j. Procedures to follow when errors, such as 
discrepant or omitted data, are discovered 

□ 

□ 

□ 

□ 

□ 


*30. How often does your district typically provide professional development for those 
administering the home language survey? 

□ Once a year 

□ Twice a year 

□ Every 2-3 years 

□ Every 4-5 years 

□ No set schedule; provided when needed 

□ Professional development is not provided 

□ I don’t know 

□ Other (please specify) 
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*31. To what extent do district personnel supervise and support those who administer and 
manage your district’s home language survey? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

32. Before moving on, are there other comments, questions, or suggestions you would like 
to make about the personnel involved in the collection, management, and reporting 
of your district’s home language survey? (optional) 


5. Data management 

The questions in this section ask about data management for the home language survey. 
Data management factors that can affect data quality include monitoring systems, coordi- 
nation and communication, storage, and accessibility. 

*33. To what extent do district personnel actively monitor the quality of data from your 
district’s home language survey? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*34. In what form are responses to your district’s home language survey stored? 

□ Only in electronic files or data system 

□ Only in paper files (if “only in paper files,” skip to question 37) 

□ Both in electronic files or data system and paper files 

*35. To what extent do the response options provided on your district’s home language 
survey directly align to the response options available in your district’s electronic files 
or data entry system? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*36. To what extent do personnel responsible for entering data from your district’s home 
language survey have adequate and uninterrupted time and space to enter data into 
electronic files or data systems? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 
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*37. To what extent are the following personnel involved in decisions about how data 
from your district’s home language survey are collected, entered, or reported? 



To no 
extent 

To little 
extent 

To some 
extent 

To great 
extent 

1 don't 
know 

District technology/data managers 

□ 

□ 

□ 

□ 

□ 

District English learner program coordinator 

□ 

□ 

□ 

□ 

□ 

School English learner specialists 

□ 

□ 

□ 

□ 

□ 

District student registration/enrollment staff 

□ 

□ 

□ 

□ 

□ 


*38. To what extent does your district maintain the confidentiality of data from the home 
language survey? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*39. To what extent can appropriate school and district personnel access data from your 
district’s home language survey in an easy, straightforward manner? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*40. To what extent can appropriate school and district personnel access data from your 
district’s home language survey in a timely manner? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*41. Does your district place responses to the home language survey in students’ perma- 
nent records or cumulative folders? 

□ Yes 

□ No 

□ I don’t know 

42. Before moving on, are there other comments, questions, or suggestions you would 
like to make regarding data management with your district’s home language survey? 
(optional) 
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Closing questions 


Two final summary questions: 

*43. Overall, to what extent is your district’s home language survey administered consis- 
tently for every student entering the district? Consider consistency throughout the 
school year and across schools in your district. 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

*44. To what extent do you feel confident about the accuracy of the responses you have 
provided throughout this survey? 

□ To no extent 

□ To little extent 

□ To some extent 

□ To great extent 

□ I don’t know 

Closing 

We appreciate your giving time to share your observations and experiences with your dis- 
trict’s home language survey. 

Please direct any questions or inquiries pertaining to this Home Language Survey Data 
Quality Self-Assessment to: 

<Insert name and contact information> 

For your information 

This Home Language Survey Data Quality Self-Assessment was developed by Regional Edu- 
cational Laboratory Northeast & Islands. Items are based primarily on the following sources: 

• Recommendations in National Forum for Education Statistics. (2007). Forum cur- 
riculum for improving education data: A resource for local education agencies (NFES 
2007-808). Washington, DC: U.S. Department of Education, National Center for 
Education Statistics. http://eric.ed.gov/lkHED497512 

• Linquanti, R., & Bailey, A. (2014). Reprising the home language survey: Summary 
of a national working session on policies, practices, and tools for identifying poten- 
tial English learners. Washington, DC: Council of Chief State School Officers. 
Retrieved March 14, 2016, from http://www.ccsso.org/Documents/2014/CCSSO%20 
Common%20EL%20Definition%20Reprising%20the%20Home%20Language%20 
Survey%2001242014.pdf. 

• Reviews by survey methodology and content experts and district practitioners. 
Survey respondents might also like to review U.S. Department of Education. 
(2015). Tools and Resources for Identifying All English Learners. In English Learner 
Tool Kit for State and Local Education Agencies (SEAs and LEAs). Washington, DC: 
Author. http://eric.ed.gov/lkHED564264 
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Appendix A. Developing the Home Language 
Survey Data Quality Self-Assessment 


Working directly with stakeholders in the region, the Regional Educational Laboratory 
(REL) Northeast & Islands project team used an iterative process to develop and refine the 
Home Language Survey Data Quality Self-Assessment. There were three main steps to the 
process: 

• Developing the initial self-assessment. 

• Conducting a small-scale administration of the self-assessment. 

• Refining the self-assessment. 

Developing the initial self-assessment 

The REL Northeast & Islands English Language Learners Alliance assembled a small 
stakeholder group focused on understanding the challenges districts face in collecting and 
reporting English learner student-focused data. This stakeholder group initially examined 
the home language survey as an example of the challenges in collecting data and identified 
several areas in the administration, documentation, and reporting of the data that could 
be improved to ensure greater reliability. 

Based on this initial examination of the home language survey, the REL Northeast & 
Islands project team then supported the stakeholder group in developing an initial self-as- 
sessment by consulting the work of other groups, including the Data Quality Campaign 4 
and the National Forum on Education Statistics. In particular, the project team drew on 
recommendations in Forum curriculum for improving education data: A resource for local 
education agencies (National Forum for Education Statistics, 2007) to develop a data quality 
framework highlighting key areas that affect data quality. This framework provided guid- 
ance to the stakeholder group and became the structure for the Home Language Survey 
Data Quality Self-Assessment. 

The development of survey items within each key area were further informed by Reprising 
the home language survey: Summary of a national working session on policies, practices, and 
tools for identifying potential English learners (Linquanti and Bailey, 2014). For example, in 
response to Linquanti and Bailey’s summary consideration, “The purposes and intended 
uses of the HLS [home language survey] should be made explicit to those administering 
and those completing the survey” (p. 3), the project team created self-assessment items to 
gather information on the extent to which districts explicitly address the purposes and 
uses of their home language survey. Items target this practice in relation to several areas of 
the data quality framework: district guidelines (items 9 and 10), parent/guardian communi- 
cations during data collection (items 19 and 20), and personnel training and support (item 
29). 

Conducting a small-scale administration of the self-assessment 

The members of the Connecticut Administrators of Programs for English Language 
Learners and the Rhode Island English as a Second Language Network conducted a vol- 
untary administration of the first version of the Home Language Survey Data Quality 
Self-Assessment in their respective states. Members of these two organizations serve as 
participants in the REL Northeast and Islands English Language Learners Alliance and 
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participated as a part of the stakeholder group. They collected 23 completed self-assess- 
merits from Connecticut and eight from Rhode Island and shared the results with the 
stakeholder group for analysis and discussion. Using the data quality framework, Rhode 
Island and Connecticut members identified two to four areas of potential concern and 
prioritized one or two of these for recommended action in their respective states. Members 
of the stakeholder group indicated that the self-assessment tool and the facilitated review 
process provided useful information and sparked questions, insights, and cross-state sharing 
of resources as well as future agendas for improving the quality of data on English learner 
students through revised guidelines and practices. 

Refining the self-assessment 

After reviewing the Rhode Island and Connecticut results, the project team and the 
English Language Learners Alliance agreed that the self-assessment tool would be useful 
to stakeholders beyond Connecticut and Rhode Island. 

To refine the initial tool, the project team first conducted a close, systematic review of 
the original survey items in relation to the responses gathered from the tool’s first use in 
Connecticut and Rhode Island. Second, the project team formed a six-member advisory 
committee with members from state education departments and school districts in Massa- 
chusetts, New Hampshire, and New York. Third, this committee provided perspectives on 
the survey from contexts that extended beyond the tool’s original use in Connecticut and 
Rhode Island. In addition, the advisory committee recommended English learner program 
coordinators from districts in their states with a low incidence of English learner students 
in their populations and those with a high incidence for participation in a series of cogni- 
tive interviews (Beatty & Willis, 2007). 5 Fourth, the project team consulted methodolog- 
ical and content experts, who provided suggestions pertaining to the particular aspects of 
survey design and the current home language survey context as represented in research 
and in the field. 

After several rounds of revisions, the project team again solicited responses to the revised 
self-assessment tool from the consulting experts and advisory committee members. The 
revision process also led to the development of a matrix detailing the alignment among 
the final survey items in relation to the four areas described in the data quality framework. 
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Appendix B. The data quality framework 


The data quality framework describes four mutually reinforcing areas that affect data 
quality: purposes, policies, and guidelines; data collection practices; personnel support; 
and data management (figure Bl ). 6 For administration of a home language survey to yield 
high-quality data, district practices in each area should be sound and aligned. A deficit in 
one area may undermine efforts in another. Collaboration among local experts and practi- 
tioners can bring diverse experience and knowledge to the design of home language survey 
practices across these four areas. 

Area 1: Purposes, policies, and guidelines 

Guiding question: Has the district established clear purposes, procedures, and guidelines to guide 
high-quality data collection during administration of the home language survey 1 

Purpose, policies, and guidelines provide the foundation for high-quality data by specify- 
ing procedures and expectations that promote common understandings and practices for 
home language survey administration within and across school districts. Data challenges 
arise from lack of clarity around the purpose of data collection, including issues with over- 
lapping or competing purposes, as well as lack of clarity about who will use the data and 
for what purposes. It may also be important for guidelines to specify how the data will not 
be used (for example, for determining legal status or reporting for immigration purposes). 
Lack of clarity of purpose can lead to instruments in which the core concepts being mea- 
sured are poorly defined, differentiated, or understood. Further, each level of the system — 
the state, district, and school — needs clarity about who is responsible for each task when 
collecting and entering data on English learner students. For example, it is assumed that 
districts follow their state guidelines regarding the home language survey, but districts 


Figure Bl. Data quality framework on which the Home Language Survey Data 
Quality Self-Assessment is based 



Source: Authors' construction based on National Forum for Education Statistics (2007) and Linquanti and 
Bailey (2014). 


B-l 


should still provide their own guidelines that reiterate and explain relevant state guidance 
to administrators of home language surveys without expecting that staff will be able to 
locate their state guidelines, will be familiar with the state guidelines, or will interpret the 
state guidelines in the same way. 

Area 2: Data collection practices 

Guiding question: Does the district ensure that its home language survey guidelines are applied 
during design and administration of the home language survey ? 

Data collection practices are defined as the application of prescribed procedures and 
expectations in designing and administering the home language survey. However, there 
are often challenges related to collection practices for home language survey data. For 
example, challenges may arise from the design of the instrument. If the phrasing of survey 
items stimulates feelings of vulnerability in respondents, the resulting data may be incom- 
plete or may be inaccurate or unreliable. Data quality challenges can also occur when the 
design of a survey varies across locations or when the instrument is not provided in an 
appropriate array of language versions. In addition, not making the purpose, use, and con- 
fidential nature of survey data explicit to participants can also reduce data quality, as can 
not maintaining consistent conditions across contexts or administrations of the survey. 
Any of these issues can lead to unreliable responses from participants. 

Area 3: Personnel support 

Guiding question: Does the district ensure that personnel who administer the home language 
survey have the necessary knowledge, skills, and support to collect high-quality data? 

To collect high-quality data, personnel must be prepared and supported to carry out proce- 
dures and expectations established by the district that are intended to promote high-qual- 
ity data. Ensuring high-quality data can be challenging when those designing instruments 
and procedures for data collection, conducting surveys or interviews, or entering or report- 
ing data — at all levels (state, district, and school) — are not consistently trained and sup- 
ported. Training and support include ensuring that personnel understand and can convey 
the intended purpose and use of the data collected, the proper procedures (including those 
pertaining to confidentiality), and how the data will affect English learner students. When 
personnel involved with data collection are not provided with resources and regular oppor- 
tunities to update or maintain their knowledge and skills through training and communi- 
cation, data quality can be undermined. 

Area 4: Data management 

Guiding question: Does the district ensure that data management systems and practices contrib- 
ute to high-quality data? 

Data management — that is, the procedures and expectations established for entering, 
archiving, and reporting data — can ensure that data quality is maintained after the initial 
data collection. Data can be entered into a system in varied ways. A respondent might 
enter responses directly into a database, or an intermediary might enter the data. The 
various personnel, sites, and schedules for data collection require periodic quality checks, 
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as well as effective coordination and communication. Good data management practices 
include establishing data collection, entry, and reporting timelines and ensuring easy and 
timely access to the data for those involved at various points. Data management is typical- 
ly enhanced when data are linked to student identification numbers in ways that can facil- 
itate monitoring and communicating with others over time. Involving technology staff 
in data management decisions can also strengthen the quality and usefulness of data and 
bring greater attention to ensuring adequate and comfortable spaces where data collection 
and entry tasks can be completed without interruption. 
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Appendix C. How items in the Home Language Survey Data 
Quality Self-Assessment align with the data quality framework 


The matrix in table Cl illustrates how items in the Home Language Survey Data Quality 
Self-Assessment align with the four areas in the data quality framework. Items addressing 
the same subtopic are aligned horizontally in the matrix and are not necessarily listed in 
numerical order. Items are organized to show how the same subtopic is addressed by differ- 
ent items in each of the four framework areas. 


Table Cl. Matrix of the alignment of items in the Home Language Survey Data Quality Self-Assessment 
with the data quality framework 


Subtopic 

Purposes, policies, 
and guidelines 

Data collection practices 

Personnel support 

Data management 


Data quality framework guiding question 


Has the district established 


Does the district ensure 



clear purposes, 

Does the district ensure 

that personnel who 



procedures, and guidelines 

that its home language 

administer the home 

Does the district ensure 


to guide high-quality 

survey guidelines are 

language survey have the 

that data management 


data collection during 

applied during design and 

necessary knowledge, 

systems and practices 


administration of the home 

administration of the home 

skills, and support to 

contribute to high-quality 


language survey? 

language survey? 

collect high-quality data? 

data? 

1. District 

*7. Has your district 

*24. Does your district 

*29e. To what extent does 

— 

attends to 

established guidelines for 

provide personnel who 

your district ensure that 


appropriate 

administering the home 

administer the home 

personnel who administer 


procedures and 

language survey? 

language survey with 

the home language survey 


expectations 

*8. In what ways does your 
district convey guidelines 
for administering the home 
language survey? 

standardized, simple-to- 
use scripts to verbally 

introduce and administer 

the survey? 

have adequate and 
current knowledge about 
administration procedures 
of the home language 
survey? 



*13. To what extent do 

*23. Does your district 

*29h. To what extent does 

*38. To what extent does 


your district's guidelines 

provide personnel who 

your district ensure that 

your district maintain the 


clearly address procedures 

administer the home 

personnel who administer 

confidentiality of data 


and responsibilities 

language survey with a 

the home language 

from the home language 


for protecting the 

frequently asked questions 

survey have adequate 

survey? 


confidentiality of data 

(FAQs) sheet to accompany 

and current knowledge 



from the home language 

the home language survey 

about confidentiality of 



survey? 

form? 

home language survey 



responses? 


(continued) 


C-l 



Table Cl. Matrix of the alignment of items in the Home Language Survey Data Quality Self-Assessment 
with the data quality framework (continued) 

Subtopic 

Purposes, policies, 
and guidelines 

Data collection practices 

Personnel support Data management 

2. District 

*14. To what extent do 

*22. To what extent 

*29c. To what extent does — 

attends to clear 

your district's guidelines 

does your district ensure 

your district ensure that 

and explicit 

clearly address how to 

that respondents to the 

personnel who administer 

home language 

interpret responses on the 

home language survey 

the home language survey 

survey purposes 

home language survey to 

are informed about the 

have adequate and current 


identify potential English 

multiple steps in the 

knowledge about how to 


learner students? 

process of identifying 
English learner students? 

use home language survey 
data as one step in a 
larger, multistep process 
for identifying English 

learner students 


*9. To what extent do your 

*19. To what extent does 

*29a. To what extent does — 


district's guidelines clearly 

your district ensure that 

your district ensure that 


address the purposes 

respondents are informed 

personnel who administer 


and intended uses of the 

about the purposes 

the home language survey 


home language survey 

and uses of the home 

have adequate and current 


(for example, identifying 

language survey? 

knowledge about the 


potential English learner 

*21. If embedded in a 

purposes or purposes 


students for subsequent 

larger intake or registration 

of the home language 


assessment to determine 

packet, to what extent 

survey? 


English learner student 

does your district 

*29d. To what extent does 


status)? 

distinguish the purpose of 
the home language survey 
from the purpose of the 
other registration or intake 

materials? 

your district ensure that 
personnel who administer 
the home language survey 
have adequate and current 
knowledge about the 
importance of high-quality 
data on English learner 
students for allocating 

resources and services? 


*10. To what extent do 

*20. To what extent does 

*29b. To what extent does — 


your district's guidelines 

your district ensure that 

your district ensure that 


clearly address what the 

respondents are informed 

personnel who administer 


purposes and intended 

about what the purposes 

the home language survey 


uses of the home 

and intended uses of the 

have adequate and current 


language survey are not 

home language survey 

knowledge about what the 


(for example, determining 

are not (for example, 

purpose and use of the 


immigration status or 

determining immigration 

home language survey 


predetermining the 

status or predetermining 

are not (for example, 


provision of education 

the provision of education 

determining immigration 


services)? 

services)? 

status or predetermining 
the provision of education 
services)? 


(continued) 
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Table Cl. Matrix of the alignment of items in the Home Language Survey Data Quality Self-Assessment 
with the data quality framework (continued) 

Subtopic 

Purposes, policies, 
and guidelines 

Data collection practices 

Personnel support 

Data management 

3. District 

attends to 

language 

support 

*15. To what extent 

does your district have 

translations of the home 
language survey in every 
language needed by survey 
respondents? 

*26. To what extent does 

your district ensure that 
respondents are provided 
with a translation of the 

home language survey at 
the time of completion, as 

needed? 

*29f. To what extent does 

your district ensure that 
personnel who administer 
the home language survey 
have adequate and current 
knowledge about cultural 
competency, or aligned 
behaviors, attitudes, 
and policies that enable 
effective work in cross- 

cultural situations? 



*16. To what extent are 

your district's translations 
of the home language 
survey created or vetted by 
professional translators? 

*25. To what extent does 

your district ensure that 
administration scripts are 
read aloud by personnel 
with sufficient foreign 
language fluency for 
respondents who need 

information translated? 

*29g. To what extent does 
your district ensure that 
personnel who administer 
the home language survey 
have adequate and 
current knowledge about 
assessing respondents’ 

needs for written 

translation or verbal 

interpretation support? 


4. District 

attends to 

oversight 
and support 
for home 

language survey 
personnel 



*29. To what extent does 

your district ensure that 
personnel who administer 
the home language survey 
have adequate and current 
knowledge about the 
following topics? [specific 
topics listed in relevant 
areas of this matrix] 





*30. How often does your 
district typically provide 
professional development 
for those administering the 
home language survey? 





*31. To what extent 

do district personnel 
supervise and support 

those who administer and 

manage your district’s 
home language survey? 


(continued) 
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Table Cl. Matrix of the alignment of items in the Home Language Survey Data Quality Self-Assessment 
with the data quality framework (continued) 

Subtopic 

Purposes, policies, 
and guidelines 

Data collection practices 

Personnel support 

Data management 

5. District 

attends to 

appropriate 
data entry, 
management, 
and reporting 
procedures 

*12. To what extent do 

your district's guidelines 
clearly address how and 
when data from the home 

language survey will 
be reported to school, 
district, and, if applicable, 

state offices? 


*29i. To what extent does 

your district ensure that 
personnel who administer 
the home language survey 
have adequate and current 
knowledge about data 
entry, management, and 
reporting responsibilities? 

*35. To what extent do the 

response options provided 
on your district's home 
language survey directly 
align to the response 
options available in your 

district’s electronic files or 

data entry system? 





*36. To what extent do 

personnel responsible for 
entering data from your 
district's home language 
survey have adequate and 
uninterrupted time and 
space to enter data into 

electronic files or data 

systems? 





*37. To what extent are the 

following personnel involved 
in decisions about how 

data from your district's 
home language survey 
are collected, entered, 
and reported: district 
technology/data managers, 
district English learner 
program coordinator, school 
English learner specialists, 

and district student 

registration/enrollment 

staff? 





*41. Does your district 
place responses to the 
home language survey 
in students' permanent 

records or cumulative 

folders? 

6. District 

attends to 

appropriate data 

access 




*34. In what form are 

responses to your district’s 
home language survey 

stored? 





*39. To what extent can 

appropriate school and 
district personnel access 
data from your district's 
home language survey in 
an easy, straightforward 

manner? 





*40. To what extent can 

appropriate school and 
district personnel access 
data from your district's 
home language survey in a 
timely manner? 

(continued) 
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Table Cl. Matrix of the alignment of items in the Home Language Survey Data Quality Self-Assessment 
with the data quality framework (continued) 

Subtopic 

Purposes, policies, 
and guidelines 

Data collection practices 

Personnel support 

Data management 

Overall. District 

attends to data 

quality protocols 

18. Before moving on, 
are there comments, 
questions, or suggestions 
you would like to make 
about your district’s 
policies and guidelines 
for the home language 
survey? 

28. Before moving on, 
are there comments, 
questions, or suggestions 
you would like to make 
about your district's data 
collection practices for the 
home language survey? 

32. Before moving on, are 
there other comments, 
questions, or suggestions 
you would like to make 
about the personnel 
involved in the collection, 
management, and 
reporting of your district's 
home language survey? 

42. Before moving 
on, are there other 
comments, questions, or 
suggestions you would like 
to make regarding data 
management with your 
district’s home language 
survey? 


*17. To what extent does 

your district specify a 
systematic process for 
periodic review of the 
quality of data from the 
home language survey? 

*11. To what extent do 

your district's guidelines 
clearly address what to 
do when an error, such 
as discrepant or omitted 
data, is discovered in data 
from the home language 
survey? 

*27. To what extent do 

any school or district 
personnel need to follow 
up with respondents 
to clarify or complete 

information submitted 

on your district's home 
language survey? 

*29j. To what extent does 
your district ensure that 
personnel who administer 
the home language survey 
have adequate and 
current knowledge about 
procedures to follow when 
errors, such as discrepant 
or omitted data, are 

discovered? 

*43. Overall, to what 
extent is your district's 
home language survey 
administered consistently 
for every student entering 

the district? Consider 

consistency throughout 
the school year and across 
schools in your district. 

*33. To what extent do 

district personnel actively 
monitor the quality of data 
from your district's home 
language survey? 





*44. To what extent 

did you feel confident 
about the accuracy of 
the responses you have 
provided throughout this 
survey? 

* Indicates that the item is required to be answered; — indicates that a relevant item was not included. In some cases a relevant item 
may have been developed but was not included as a critical element in the final self-assessment tool, which was reviewed, in part, for 
brevity and potential completion within 15 minutes. 

Note. This matrix was constructed by the project team to help users identify how self-assessment items align with the four data quality 
framework areas (columns) and with other items representing the same cross-cutting subtopics (rows). These subtopics were identified 
by the project team based on themes identified in the two primary sources consulted during framework development (see appendix A), 
as well as feedback collected during cognitive interviews with district English learner program coordinators. 

Source: Authors' creation. 
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Appendix D. Workshop protocol: Working with the results 
of the Home Language Survey Data Quality Self-Assessment 


The following agenda and protocol describe one way that states can engage stakeholders 
in examining and taking action on data collected through the Home Language Survey 
Data Quality Self-Assessment. This process should be modified to suit varied contexts, 
stakeholder groups, and timeframes. 

Session participants 

Session facilitator (for example, the state education agency lead or an independent facili- 
tator), state education agency leader, and 5-10 state or district stakeholders (for example, 
state leaders, district and school administrators, English learner program coordinators and 
specialists, teachers, parents/guardians, data managers, or district student registration staff). 

Session purpose 

To identify and prioritize areas for improving the quality of data from the home language 
survey and consider application of lessons to other efforts to collect data on English learner 
students. 

Session goals 

Participants will: 

• Discuss a framework for understanding the quality of data from the home lan- 
guage survey. 

• Examine the results from the Home Language Survey Data Quality Self-Assess- 
ment to identify strengths and weaknesses in administration across the state. 

• Identify opportunities for improvement and prioritize areas for action. 

• Identify action steps and key lessons for strengthening data quality. 

Session outcome 

Agree on next steps to address ways to support the collection of high-quality data from the 
home language survey. 


Session agenda 


1 Time 

Activity 

Focus question 1 

9:00 a.m. 

Welcome and session overview 

Why are we focusing on data quality? 

9:10 a.m. 

Introduce the data quality framework 

How do the items on the self-assessment align 
with the data quality framework? 

9:40 a.m. 

Frame the inquiry 

What are the focus and limitations of the data? 

10:00 a.m. 

Analyze results from the Home Language 
Survey Data Quality Self-Assessment 

What strengths and weaknesses do we notice in 

our data? 

11:30 a.m. 

Identify opportunities for improvement 
and prioritize areas for action 

What actions can we take to gain the greatest 
improvements in the quality of data from the 
home language survey? 

12:15 p.m. 

Identify generalizable lessons 

What have we learned that applies to other efforts 
to collect data on English learner students? 

12:30 p.m. 

Closing 
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Facilitator preparation 

• Coordinate session time with participants, meeting room, and refreshments. 

• Determine who will facilitate the session. 

• Prepare slide deck: 

o Slide 1: Participants and affiliation. 

° Slide 2: Session purpose. 

o Slide 3: Session goals and outcome. 

o Slide 4: Session agenda. 

° Slide 5: Data quality framework graphic (see figure B1 in appendix B). 

° Slide 6: Response rate statistics from the Home Language Survey Data Quality 
Self-Assessment (bulleted list including percentage of districts responding, 
percentage of districts responding across the state, percentage of districts with 
a low, medium, and high incidence of English learner students in their popu- 
lation responding, size of responding districts, and other pertinent data about 
respondents). 

° Slide 7: Copy of Handout 5: Individual data review sheet (optional). 

• Prepare handouts for all participants and charts as noted below: 

o Handout 1: Data quality framework graphic and self-assessment items by 
section (provided below). 

o Handout 2: Description of the data quality framework (see appendix B). 

o Handout 3: Home Language Survey Data Quality Self-Assessment (blank 
copy). 

o Handout 4: Summary results from Home Language Survey Data Quality 
Self-Assessment (provided by the state, with results presented by item in sim- 
ple-to-read, clearly organized graphs and tables). 

o Handout 5: Individual data review sheet (provided below). 

o Handout 6: Alignment matrix (see table Cl in in appendix C). 

° Chart 1: Prepare four charts, each with sections for recording: framework area, 
factors that affect data quality, challenges to data quality, and questions; see 
9:10-9:40 a.m. for further description of activity. 

o Chart 2: Prepare one chart with the following sections: response rate limita- 
tions, data summary limitations, and how to determine areas of strength and 
weakness. 

° Chart 3: Prepare four charts with the following sections (consider providing 
one chart for each topic): framework area, strengths (include item numbers 
and evidence), weaknesses (include item numbers and evidence), ques- 
tions that might be answered by further data analysis, and opportunities for 
improvement. 

° Chart 4: Prepare one three-column chart with headings “Priorities,” “Action 
steps,” and “Responsible party” at the top of each column. 
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Process agenda 


9:00-9:10 a.m. 


Agenda item 

Welcome and session overview. 

Purpose 

To set a welcoming tone and provide rationale for the day’s work. 

Focus question 

Why are we focusing on data quality? 

Materials 

• Slide 1: Participants and affiliation. 

• Slide 2: Session purpose. 

• Slide 3: Session goals and outcome. 

■ Slide 4: Session agenda. 

Steps: Facilitator 

(5 minutes) Introduces self and welcomes other stakeholders at the table. Asks each to introduce himself or 
herself, telling name, role, affiliation, and connection to the English learner identification process. (Facilitator 
may provide brief meet-and-greet activity and adjust time accordingly if participants have not previously 
worked together.) 

(3 minutes) Explains challenges with data quality based on literature and notes examples from state or district 
context, for example: 

■ Recent research points to multiple factors that often undermine home language survey data quality, 
including unclear survey purposes, lack of appropriate language supports for parents to complete the 
survey, untrained staff administering the survey, inconsistent practices during survey administration, and 
inaccurate data entry (Bailey & Kelly, 2010, 2013; Linquanti & Bailey, 2014). Any of these may contribute 
to poor data quality, which can result in the misidentification of potential English learner students in school 
districts. Misidentification of students poses a challenge for students and their families as well as districts 
and state education departments as they strive to appropriately allocate resources and services to support 
English learner students. 

(2 minutes) Provides brief overview of session purpose, goals, and outcomes, then the session agenda 
(Slides 1-4): 

Today’s purpose: 

•> Identify and prioritize areas for improving the quality of data from the home language survey and 
consider application of lessons to other efforts to collect data on English learner students. 

Today’s goals: 

° Discuss a framework for understanding the quality of data from the home language survey. 

° Examine the results from the Flome Language Survey Data Quality Self-Assessment to identify strengths 

and weaknesses in administration across the state. 

° Identify opportunities for improvement and prioritize areas for action. 

° Identify action steps and key lessons for improving data quality. 

Today’s outcomes: 

° Agree on next steps to address the collection of high-quality data from the home language survey. 

Today’s agenda: (note that the group will take a break between 10:00 and 11:30) 
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9:10-9:40 a.m. 


Agenda item 

Introduce the data quality framework. 

Purpose 

To become familiar with four key areas that can affect data quality and how the self-assessment gathers 

information in these areas. 

Focus question 

Flow do the items on the self-assessment align with the data quality framework? 

Materials 

• Slide 5: Data quality framework graphic (see figure Bl). 

Flandout 1: Data quality framework and self-assessment items by section. 

• Flandout 2: Description of the data quality framework (appendix B). 

Flandout 3: Flome Language Survey Data Quality Self-Assessment (blank copy). 

• Chart 1 (four copies) and markers. 

Steps: Facilitator 

(10 minutes) Introduces the data quality framework by referencing Slide 5: Data Quality Framework graphic and 
by distributing Flandout 1: Data quality framework graphic and self-assessment items by section. 

Emphasizes that the framework area of purposes, policies, and guidelines refers to the district’s specification 
of this guidance; the framework area of data collection practices refers to the application of the district’s 
guidance; the framework area of personnel support refers to the necessary knowledge and skills for staff to 
apply the district’s guidance; and the framework area of data management refers to the maintenance of the 

data collected. 

Divides the group into small groups (or pairs or individuals, depending on the size of the full group) and assigns 
each small group one of the framework areas. Distributes chart paper, markers, and Flandout 2: Description of 
the data quality framework description (appendix B). Asks participants to read about their assigned framework 
area, then record on Chart 1: 

The name of the framework area. 

A bulleted list of the key factors that affect data quality in that category, based on the handout description. 

(5 minutes) Prompts small groups to rotate to another small group’s chart to review the chart (and the data 
quality framework, as needed), then add onto it by recording: 

• Challenges to the quality of data from the home language survey that they have seen, heard, or might 
imagine in relation to this category. 

(5 minutes) Prompts small groups to rotate to another small group’s chart to review the chart (and data quality 
framework, as needed) and challenges listed, then add on by recording: 

■ Questions they have about how to ensure home language survey data quality in this area. 

(5 minutes) Prompts small groups to rotate back to their original chart, then review the items in the focal area 
from the Flome Language Survey Data Quality Self-Assessment by referring to Flandout 1 and Flandout 3: The 
Flome Language Survey Data Quality Self-Assessment (blank copy), then record: 

• Checkmarks on bulleted factors, challenges, and questions that self-assessment items address or partially 

address. 

(5 minutes) Describes the checkmarked topics across the charts that represent issues identified by recent 
research or expertise from the field. Solicits observations and responses from the full group, then notes that 
the self-assessment items serve as informal guidance about ways that states and districts can strengthen 
the quality of home language survey data (for example, provision of administration scripts, training on specific 
topics, and data management strategies). 
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9:40-9:55 a.m. 


Agenda item 

Frame the inquiry. 

Purpose 

To understand the limitations of data from the Home Language Survey Data Quality Self-Assessment prior to 
using the data. 

Focus question 

What are the focus and limitations of the data? 

Materials 

• Slide 6: Response rate statistics from the Home Language Survey Data Quality Self-Assessment. 

Handout 4: Summary results from the Home Language Survey Data Quality Self-Assessment (summary 
statistics and any desired disaggregated data). 

• Chart 2 (one copy) and markers. 

Steps: Facilitator 

(10 minutes) Explains that the group will be discussing strengths and weaknesses that are suggested by the 
self-assessment results. Shows and describes information on Slide 6: Response rate statistics from the Home 
Language Survey Data Quality Self-Assessment. For example, 55 percent of districts in the state responded, 
plus some descriptive statistics about which districts responded, such as 25 percent of respondents were 
from high-incidence districts, 35 percent from medium-incidence districts, and 40 percent from low-incidence 

districts. 

Asks the group to describe what the response rate suggests about limitations to the group's interpretation of 
the results. For example, high-incidence districts are under-represented in the target population, so the group 
needs to be cautious about applying interpretations from the data to these districts. Records responses on 

Chart 2. 

Asks group to view the data summary and describe what the data include (for example, percentage of 
respondents who indicated the extent to which a data practice occurs in the district; open responses with 
additional perspective) and what the data do not include (for example, summary statistics for each category 
in the framework or disaggregated views by subgroup). Records responses on Chart 2. Explains that data can 
be disaggregated after the session, or in preparation for a follow-up session, if the group identifies specific 
questions for inquiry during the day's work. 

(5 minutes) Asks group how they will define areas of strength (for example, more than 70 percent indicate 
to some or great extent) and areas of weakness (for example, more than 40 percent indicate to no or little 
extent). Asks the group to test these criteria against a couple of sample items in the results to see if all agree 
that these outpoints make general sense. Explains that these criteria need to be somewhat flexible, depending 
on the item and the group's ongoing discussion and are meant only to facilitate the group’s inquiry and 

discussion. Records decisions on Chart 2. 


D-5 


9:55-11:35 a.m. 


Agenda item 

Analyze results from the Flome Language Survey Data Quality Self-Assessment. 

Purpose 

To identify practices that both contribute to and undermine home language survey data quality across the 

state. 

Focus question 

What strengths and weaknesses do we notice in our data? 

Materials 

• Slide 7: Copy of Flandout 5: Individual data review sheet. 

Flandout 4: Summary results from Home Language Survey Data Quality Self-Assessment. 

• Handout 5: Individual data review sheet. 

• Chart 3 (four copies) and markers. 

Handout 6: Alignment matrix. 

Steps: Facilitator 

(5 minutes) Explains that participants will be asked to review Handout 4: Summary results from the Home 
Language Survey Data Quality Self-Assessment individually in more detail now, so the room will be quiet while 
they review and record notes on Handout 5: Individual data review sheet. Reviews directions on Handout 5 and 
responds to questions. Emphasizes that this is a quiet time to consider the results individually and that the 
group will have time to discuss and work with the results after this period. 

(20 minutes) Individual work 

(5 minutes) Divides the group into small groups (or pairs or individuals, depending on the size of the full group) 
and assigns each small group one or two areas in the self-assessment (purposes, policies, and guidelines; 
data collection practices; personnel support; or data management) to delve into. Explains that each small 
group will refer to the relevant sections of their completed Handout 5 to compare and discuss observations, 
one item at a time, and to chart identified strengths and weaknesses in each assigned area, making sure to 
note the specific numbers of relevant self-assessment items and evidence from the data and basing these 
strengths and weaknesses on criteria set earlier in the group’s work (9:40-9:55 a.m.). 

Encourages participants to engage in probing, evidence-based discussion: ask colleagues why they identified 
areas as strengths and weakness, whether they think the results are expected or surprising and why, and what 
questions the data raise for them. In particular, encourages each small group to record questions that might 
be answered by disaggregating the self-assessment results and opportunities for improvement identified on 
the basis of their analysis. Explains that each small group will be asked to report out to the full group, using 
their chart as a visual guide. Solicits and responds to questions. 

(30 minutes) Teamwork. 

(10 minutes) Break. 

(10 minutes) Distributes Handout 6: Alignment matrix and explains howto read the chart: if participants look 
down the columns, they will see how all self-assessment items are aligned with specific framework areas, 
and if they look across the rows, they will see how self-assessment items align with key subtopics that run 
throughout the self-assessment. Allows several minutes for participants to review the matrix and responds to 
questions. 

Explains that during the small-group report-outs, participants can mark on their matrix items associated with 
strengths and weaknesses (for example, placing a checkmark on the items identified as strengths and an X 
on the items identified as weaknesses); notes that the full group will refer to these notes again after the break 
when they work to identify priorities for action. 

(20 minutes) Facilitates small-group report-outs to full group (three to five minutes per small group to describe 
their charted strengths and weaknesses, questions, and opportunities for improvement) and solicits clarifying 
questions. 
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11:35 a.m.-12:25 p.m. 

Agenda item Identify opportunities for improvement and prioritize areas for action 


Purpose 

To develop shared commitment to a strategic and limited set of priorities to strengthen the quality of data from 
the home language survey. 

Focus question 

What actions can we take to gain the greatest improvements in the quality of data from the home language 
survey? 

Materials 

• Prior small-group report-out charts (Chart 3). 

Flandout 6: Alignment matrix with recorded notes. 

• Chart paper and markers. 

Steps: Facilitator 

(5 minutes) Explains that the group now considers the opportunities for improvement described by each small 
group and identifies the highest priorities for action, first in pairs or triads, then as a large group. Explains that 
highest priority means potential high impact on data quality and feasible for the state to address within the 
coming one to two years. 

(10 minutes) Asks pairs or triads to discuss their notes on Flandout 6: Alignment matrix, and the group report- 
outs to identify up to three high priorities for state action, along with a rationale, to record on chart paper and 
share with the full group. 

(20 minutes) Asks each pair or triad to share its three priorities and provide a rationale for each. Cues the 
full group to listen for connections and for ways to achieve consensus in the group around three priorities. 
Facilitates sharing and consensus-building. Records final three priorities on Chart 4: Priorities, action steps, 
and responsible party. 

(15 minutes) Facilitates group discussion and records group ideas pertaining to key action steps for each 
identified priority and designates who would be responsible (presumably the state, but it may be the working 
group or districts). This is intended to capture the group’s consensus; it is not a formal action plan, which will 
need to be drafted and communicated following the meeting. 

12:25-12:37 p.m. 


Agenda item 

Identify generalizable lessons. 

Purpose 

To reflect on the process and results to identify key ideas or concepts that might improve other efforts to 
collect data on English learner students. 

Focus question 

What have we learned that applies to other efforts to collect data on English learner students? 

Materials 

Chart paper and markers. 

Steps: Facilitator 

(5 minutes) Solicits and records ideas from the group about what members learned from the day's work and 
how the lessons might apply to other efforts to collect data on English learner students. 

(2 minutes) Thanks participants for their time and insights. Recaps next steps agreed on during the meeting, 
and reminds participants of any upcoming meetings, communications, or other follow-up to ensure progress in 
the identified priority areas. 

(5 minutes) Provides a session evaluation form or collects feedback in the moment — for example by using a 
Plus/Delta Protocol (optional). 
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Handout 1: Data quality framework and self-assessment items by section 


Table Dl. Self-assessment items by section 

1 Item number 

Section 

Guiding question and description of items I 

1-6 

1. District context 

District size, English learner student population, and means of 
home language survey administration. 

7-18 

2. Purposes, policies, 
and guidelines 

Has the district established clear purposes, procedures, and 
expectations [guidelines?] to guide high-quality data collection 
during administration of the home language survey? 

Items address the specification of procedures and expectations 
that promote common understandings and practices for home 
language survey administration. 

19-28 

3. Data collection 

practices 

Does the district ensure that its home language survey [policies 
and?] guidelines are applied during design and administration of 
the home language survey? 

Items address the application of prescribed procedures and 
expectations, typically enacted as a sequence of events using 
provided resources. 

29-32 

4. Personnel support 

Does the district ensure that personnel who administer the 
home language survey have the necessary knowledge, skills, and 
support to collect high-quality data? 

Items address the preparation and supervision of personnel 
to ensure the necessary knowledge and skills to apply the 
prescribed procedures, expectations, and resources for home 
language survey administration. 

33-42 

5. Data management 

Does the district ensure that data management systems and 
practices contribute to high-quality data? 

Items address the use of systems, procedures, and expectations 
for entering, archiving, and reporting home language survey data. 

43-44 

6. Closing questions 

Questions about consistency of administration and perceptions 
about accuracy of results. 

Source: Authors' compilation. 
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Handout 5: Individual data review sheet 


Review the results of Home Language Survey Data Quality Self-Assessment, which solic- 
ited information from district English learner program coordinators about their local con- 
texts and their perspectives and observations in four core areas that can affect the quality 
of home language survey data: purposes, policies, and guidelines; data collection practices; 
personnel support; and data management. 

As you review each section, please note three types of responses: 

• Circle the item numbers with responses that strike you as interesting or concern- 
ing or that may provide some new insight about your state’s or district’s English 
learner student data quality. For example, you may notice notable discrepancies 
across participants’ responses for a particular item or you may identify an item 
where most responses indicate weak or overlooked data practices. 

• Place a checkmark on items suggesting potential areas of strength. For example, 
there may be general consensus around the occurrence of some positive data 
practices. 

• Jot notes, questions, or observations following each segment that may be useful to 
discuss or further explore with your team. 

1. Context 

Items: 1 2 3 4 5 6 

Notes, questions, and observations: 


2. Purposes, policies, and guidelines 

Items: 7 8 9 10 11 12 13 14 15 16 17 18 

Notes, questions, and observations: 


3. Data collection practices 

Items: 19 20 21 22 23 24 25 26 27 28 

Notes, questions, and observations: 


4. Personnel support 

Items: 29a 29b 29c 29d 29e 29f 29g 29h 29i 29j 

Notes, questions, and observations: 
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30 31 32 


5. Data management 


Items: 33 34 35 36 

Notes, questions, and observations: 


6. Closing questions 
Items: 43 44 


Notes, questions, and observations: 


Notes 


The English Language Learners Alliance played a key role over the course of the project. 

In addition, the following individuals were integral to the development and refinement of 

the tool: 

• Stakeholder group: Karen Lapuk (Connecticut), Bob Measel (formerly of Rhode 
Island), Michael Sabados (Connecticut), and Marie Salazar Glowski (Connecticut). 

• Advisory committee members: Bonnie Baer-Simahk (Massachusetts), Patrick Larkin 
(Massachusetts), Bob Measel (Massachusetts), Andrea Somoza-Norton (New 
Hampshire), Xrystya Szyjka (New York), and Deborah Wall (Massachusetts). 

• District English learner program coordinators: Karen Goyette (New Hampshire), 
Kellie Jones (Massachusetts), Kathleen Lange-Madden (Massachusetts), Nicole 
Lindeman (New York), Diane Mclver (New York), Wendy Perron (New Hamp- 
shire), and Rachel Stead (New York). 

1. An alternate title is English learner student coordinator. The most knowledgeable 
respondents should be determined locally but might include a district superintendent, 
deputy director, accountability or compliance manager, English learner program direc- 
tor, or student enrollment coordinator. 

2. To encourage more valid, or truthful, responses from parents and other caregivers com- 
pleting home language surveys, Linquanti and Bailey (2014) suggest that states specify 
what the home language survey results will not be used for (for example, determining 
immigration status or predetermining education services to be provided). 

3. Nonresponse occurs when respondents purposefully or accidentally do not complete 
and return the self-assessment or leave items blank. Bias occurs when answers pro- 
vided by those who completed the items differ from answers that would have been 
provided by those who did not respond. Nonresponse skews the results toward the 
responses provided. 

4- The Data Quality Campaign (http://dataqualitycampaign.org) is a national nonprofit 
organization that supports state policymakers and other key leaders in promoting the 
effective use of data to improve education outcomes for students. 

5. The purpose of Beatty & Willis’s (2007) in-depth cognitive interviews was to collect 
verbal information from participants about survey items while they were complet- 
ing the survey in order to determine whether the survey overall, and the items and 
response choices specifically, generated the information that the developers intend- 
ed to collect. For example, participant responses might consist of explanations of 
what they interpreted the questions to mean, elaborations on how they selected their 
responses, reports of any difficulties they had in answering items; or information about 
their school, district, or state context that shaped their responses. 

6. The framework, including the guiding questions and descriptions, is based primarily 
on recommendations in National Forum for Education Statistics (2007) and Linquanti 
and Bailey (2014). 
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The Regional Educational Laboratory Program produces 7 types of reports 



Making Connections 

Studies of correlational relationships 


Making an Impact 

Studies of cause and effect 


What’s Happening 

Descriptions of policies, programs, implementation status, or data trends 


What’s Known 

Summaries of previous research 


Stated Briefly 

Summaries of research findings for specific audiences 


Applied Research Methods 

Research methods for educational settings 


Tools 

Help for planning, gathering, analyzing, or reporting data or research 



